9 research outputs found
Message-Passing Inference on a Factor Graph for Collaborative Filtering
This paper introduces a novel message-passing (MP) framework for the
collaborative filtering (CF) problem associated with recommender systems. We
model the movie-rating prediction problem popularized by the Netflix Prize,
using a probabilistic factor graph model and study the model by deriving
generalization error bounds in terms of the training error. Based on the model,
we develop a new MP algorithm, termed IMP, for learning the model. To show
superiority of the IMP algorithm, we compare it with the closely related
expectation-maximization (EM) based algorithm and a number of other matrix
completion algorithms. Our simulation results on Netflix data show that, while
the methods perform similarly with large amounts of data, the IMP algorithm is
superior for small amounts of data. This improves the cold-start problem of the
CF systems in practice. Another advantage of the IMP algorithm is that it can
be analyzed using the technique of density evolution (DE) that was originally
developed for MP decoding of error-correcting codes
Universality for Multi-terminal Problems via Spatial Coupling
Consider the problem of designing capacity-achieving codes for multi-terminal communication scenarios. For point-to-point communication problems, one can optimize a single code to approach capacity, but for multi-terminal problems this translates to optimizing a single code to perform well over the entire region of channel parameters. A coding scheme is called universal if it allows reliable communication over the entire achievable region promised by information theory.
It was recently shown that terminated low-density parity-check convolutional codes (also known as spatially-coupled low-density parity-check ensembles) have belief-propagation thresholds that approach their maximum a-posteriori thresholds. This phenomenon, called "threshold saturation via spatial-coupling," was proven for binary erasure channels and then for binary memoryless symmetric channels. This approach provides us with a new paradigm for constructing capacity approaching codes. It was also conjectured that the principle of spatial coupling is very general and that the phenomenon of threshold saturation applies to a very broad class of graphical models.
In this work, we consider a noisy Slepian-Wolf problem (with erasure and binary symmetric channel correlation models) and the binary-input Gaussian multiple access channel, which deal with correlation between sources and interference at the receiver respectively. We derive an area theorem for the joint decoder and empirically show that threshold saturation occurs for these multi-user scenarios. We also show that the outer bound derived using the area theorem is tight for the erasure Slepian-Wolf problem and that this bound is universal for regular LDPC codes with large left degrees. As a result, we demonstrate near-universal performance for these problems using spatially-coupled coding systems
A Simple Proof of Maxwell Saturation for Coupled Scalar Recursions
Low-density parity-check (LDPC) convolutional codes (or spatially-coupled
codes) were recently shown to approach capacity on the binary erasure channel
(BEC) and binary-input memoryless symmetric channels. The mechanism behind this
spectacular performance is now called threshold saturation via spatial
coupling. This new phenomenon is characterized by the belief-propagation
threshold of the spatially-coupled ensemble increasing to an intrinsic noise
threshold defined by the uncoupled system. In this paper, we present a simple
proof of threshold saturation that applies to a wide class of coupled scalar
recursions. Our approach is based on constructing potential functions for both
the coupled and uncoupled recursions. Our results actually show that the fixed
point of the coupled recursion is essentially determined by the minimum of the
uncoupled potential function and we refer to this phenomenon as Maxwell
saturation. A variety of examples are considered including the
density-evolution equations for: irregular LDPC codes on the BEC, irregular
low-density generator matrix codes on the BEC, a class of generalized LDPC
codes with BCH component codes, the joint iterative decoding of LDPC codes on
intersymbol-interference channels with erasure noise, and the compressed
sensing of random vectors with i.i.d. components.Comment: This article is an extended journal version of arXiv:1204.5703 and
has now been accepted to the IEEE Transactions on Information Theory. This
version adds additional explanation for some details and also corrects a
number of small typo